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ABST ^ ACT 

Learner verification, a process by which publishers 
mon^-^o- th^ effectiveness of their products and strive to improve 
their services to schools, is a practice that most companies take 
seriously. The quality of educational materials may be ensured m 
many ways: by analysis of sales, through firsthand investigation, and 
by employing a system of evaluators within the school system. Two 
case studies are described, one detailing the failure of a product 
designed to aid in word recognition, the other outlining th- use of 
an independent research group to predict teacher and student _ 
acceptance of a flexible instructional packet. While the analysis of 
^educational needs and the pretesting of instructional materials may 
increase production costs, they can also prevent the proliferation of 
irrelevant products. (K5) 
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LEARNER VERIFICATION: A PUBLISHER'S CASE STUDY 

George Wilson 

r-H There is a story of an unusual insect in the last issue of Natural 
Q 

Uj History. It seems that when conditions are favorable, these gall 
midges breed and the young develop as larvae in the mother's body. 
They grow by feeding on the mother until there is nothing left but 
an empty shell. Now, that seems to be a strange story with which 
to begin a discussion of learner verification and product development. 
But, the gall midge larvae are not unlike the image some people have 
of publishers. We're often pictured as a bunch of larvae ready to 
feast on the educational community until there is nothing left but 
a shell of uneducated kids. It ' s a myth perpetuated by people like 
that testor of electric light bulbs who for ^50 offers to tell you 
which educational software tools are rip offs and which will work for 
your students. And it " s a picture of publishers that is leading to 
some questionable legislation and questionable demands on publishers. 

It's my goal today to try to present another view of publishing. It 
is a view of concerned educators providing a service to schools, ir 
business at the will of the educational community, and only in business 
so long as we provide a real service to you. The profit motive forces 
us into this role. Bacau£;e most publishers want to serve you in such 
a way as to continue in business for the years to come. 
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In recent years, rnuchlias beer, made of the "learner verification" idea. 
It has appeared in legislation in a number of states, and is an ex- 
pression of a very good idea ~ th'at publishers have a responsibility 
to constantly monitor their products and to be constantly striving to 
improve their services to schools. 

That's a re onsibility publishers have borne since they began, and one 
that most publishers take very seriously. But the present vagueness 
of "learner verification" requirements give us concern. First, because 
some of the reporting made for the sake of learner verification may be 
misleading. It would require much more massive studies than are required 
by current legislation to arrive at any high probability that any pro- 
duct would be effective in any given situation. And this kind of exam- 
ination of products probably would not be cosr effective either in terms 
of money invested by publishers . in terms of additional assurances to 
users of the materials. Statistics gathered in any form can be mislead- 
ing. Who was it who described educational statistics as numbers that 
proved that something was significant even if it wasn't important? 

Publishers go about ensuring the quality of their products in a variety 
of ways — none of which are probably "real" learner verification. The 
company I represent is a comparatively small one, but we are probably 
not atypical in the methods we use. 

First, most publishers are educators i:. a very real sense. Many people 
on publishing staffs are former teachers and administrators ... most of 
our editors and technicians are. For example, George Holland is our 
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English and humanities editor. He's a former teacher of English at 
the University of Arizona. Madeleine Shaw is a graduate of Brooklyn 
College and a former teacher in Brooklyn and in California. This 
previous educational experience lends perspective to the work, but 
beyond that, so what? You may be thinking of Sam, the old fellow 
who went to hear his friend Ed campaign for his umpteenth reelection 
to the legislature. Following a rousing speech full of political 
rhetoric, Ed felt comfortable in calling for a show of friends who 
would vote for him. Everyone raised his hand — except Sam. The 
conversation went something like this. "Well, Sam, what's wrong? 
Haven't I worked to get the sewer line to your property? Didn't 
I protect your water rights? Aren't the schools your kids went 
to good? Didn't I controJ your taxes?" "Yes," said Sam. "Then 
why didn't you vote for me?" "Well, Ed," said Sam, "what have you 
done for me lately?" 

People, even publishers are inclined to live in the present and to 
remember the extremes of the past. So it's easy, even for people 
like George and Madeleine to distort what kids can do, to redefine their 
skills in adult torrr.r: . V7o use a variety of ways to keep up with you 
today . 
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At BFA wc like to think of somcthiny wc call the Oi'A School System — 
a growing list of school people around the country in which we have 
confiden'^e and who v/e can call on for advice. We get hundreds of 
reports from these people. And we pay attention. 

But our editors like first-hand information about their brainchildren^ 
So they are constantly taking the new products out to try them in the 
schools. We formalize this a little more by lending our staff to 
schools as teacher aides part of every year. Right now, Joyce Moss 
is working in a third grade class and learning about one of our newest 
reading products- 

Most of our testing is done after the project is completed; that's 
when it can really be tested as a whole. But it creates a real 
problem for publishers that requires your help. 

Let me illustrate with a couple of histories of products we've produced. 
The first case is a product I can't show you because I don't know 
v'here it is. Three years ago, our BFA School System led us to be- 
lieve that there was a need for some proven materials to teach word 
recognition. As often happens, we discovered that need at the same 
time someone else did. In this case, it wjs a group from a reading 
clinic with a program that used color coding to instruct kids about 
clues to word recognition. And they had students available to test 
the program as it developed. It seemed like a good idea, so we went 
ahead . 
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But we suspected that we were in trouble when we took the first 
units out to see how our new product worked. In the first trial 
group of six kids, no one seemed to need the instruction. In the 
next group, one six year old followed the lesson for two or three 
minutes then just got up and left. "That's all I'm going to do, 
thank you." At least he was polite. 

So we sent the units to five people around the country to be tried 
in schools. Educators are notoriously polite — but they weren't 
about this project. So it never made the marketplace. Testing 
products can be a very expensive activity. 

Publishers ought to pay for their errors, and they do. BFA is no- 
large, but in the six years I've been associated there, we've chosen 
to lose $500,000 in production costs on projects that we found didn't 
meet our expectations . 

But consider another example. The BFA School System told us two 
years ago that classrooms were becoming more flexible. There was a 
move to accountability as never before and teachers were meeting this 
charge with a variety of teaching modes. So, we designed a program 
that could be used in a variety of ways — for class presentations, 
for small groups, for individual study in a science corner ... everything 
color keyed so that teachers and students could easily identify the 
parts needed for whatever mode . . .pre-tests, post- tests ... everything 

the teachers said they needed. 



Sometimes in testing a product we don't trust our own biases. So 
we employed an independent research group to see if the product did 
what it was designed to do. They elicited reactions from 39 teachers 
and administrators, all of whom were certain that the product was what 
they needed. The research group then placed the package in 11 classrooms 
for a month of trial. It worked: Just the way it was supposed to 



work. 



NOW, in no way could anyone claim that what we did represented a very 
formal learner verification. But the small trial we made added 24 
percent to the costs of developing the project. Sales indications 
are, so far, that we'll recover these costs about 1979. What went 
wrong? Who knows? P-rhaps the subject matter. At any rate, we've 
tried again with a slight modification of the package in a program 
called POE'iRY TICKLES. I brought this program along in case anyone 
would like to listen and look and give us a little advice. 

These two cases illustrate something we all need to consider. If 
learner verification in any form is to become a reality, someone 
will have to pay for it. At the least, schools will have to share by 
providing free access to schools for testing materials, and will have 
to express a decided preference for tested materials over untested 
materials. More formal verification will certainly result in higher 
prices, and may be very difficult to justify on the basis of cost- 
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effectiveness- After all, there is no absolute guarantee that a 
product will work under all circumstances and for all of your 
students. 

Publishers, I believe, have responsibilities to help you by ana- 
lyzing your needs before developing materials, and for testing 
materials in development if possible, and certainly for measuring 
effectiveness of the completed product. It is a lot easier task 
if you can find an author with expertise in the subject and with 
the facilities for testing the material during development. As 
we developed the Comprehension Skills Lab, Dr. Martyn and his 
team had already surveyed the reading needs and analyzed existing 
tools. Some things were obvious kids were having trouble with 
standardized tests of comprehension, and most products designed to 
help could have used a little more real instruction to go with the 
practice. The American Learning team produced what we think helps 
with these problems. The emphasis is on comprehension, with inter- 
esting new readings, and there is built in instruction — even for 
readers with very large difficulties. Now we're following up with 
field tests. We have tested on small scales and now have the kits 
in more than 100 schools for sLudy. 

I'll leave it to Dr. Martyn to tell about the problems of authors 
in the development process, but this kit also illustrates a point 
I would like to emphasize. BFA, like all publishers, is in the 
business to make a profit. Our profit comes from providing real 
services to the schools. The list of extinct companies who have 



tried and failed to provide this service is large. We began with 
the Comprehsnsion Skills Laboratory marketing in January, 1976, 
with one kit. We want to make more. We want to be in business 
for a long time — in fact, we've just been struggling with our 
plans for the next five years. As a service organization, we'll 
provide whatever service is demanded by the schools that is 
reasonab"* e . 

In the matter of learner verification, the problem is defining what 
is reasonable and possible .. .and findi.^g ways to cooperate with 
the schools to maintain costs at an effective level. 
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